Papers with test case generation
Extraction of Message Sequence Charts from Software Use-Case Descriptions (N19-2)
Copied to clipboard
Girish Palshikar, Nitin Ramrakhiyani, Sangameshwar Patil, Sachin Pawar, Swapnil Hingmire, Vasudeva Varma, Pushpak Bhattacharyya
| Challenge: | Software Requirement Specification documents provide natural language descriptions of the core functional requirements as a set of use-cases. |
| Approach: | They propose a linguistic knowledge-based approach to extract software requirements from use-cases using a textual representation of the core functional requirements. |
| Outcome: | The proposed method performs better than existing techniques and improves performance. |
TESTEVAL: Benchmarking Large Language Models for Test Case Generation (2025.findings-naacl)
Copied to clipboard
Wenhan Wang, Chenyuan Yang, Zhijie Wang, Yuheng Huang, Zhaoyang Chu, Da Song, Lingming Zhang, An Ran Chen, Lei Ma
| Challenge: | Existing methods to generate test cases using large language models are limited in their ability to generate unit test cases. |
| Approach: | They propose a test case generation benchmark that uses large language models to generate unit test cases. |
| Outcome: | The proposed test case generation benchmarks compare LLMs with commercial and open-source LLM platforms and find that they lack the ability to comprehend program logic and execution paths. |
CodeContests+: High-Quality Test Case Generation for Competitive Programming (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Competitive programming has become a key task for training and evaluating large language models . but test cases of competitive programming problems are often difficult to obtain . |
| Approach: | They propose an LLM-based agent system that creates high-quality test cases for competitive programming problems. |
| Outcome: | The proposed system improves code tests on a CodeContests dataset with pass/fail labels. |
Program Structure-aware Language Models: Targeted Software Testing beyond Textual Semantics (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models for test case generation have improved branch coverage via prompt-engineered mutations, limiting their effectiveness for discovering subtle bugs and security vulnerabilities. |
| Approach: | They propose a program structure-aware LLM framework that integrates code property graphs and code semantics to condition test case generation on execution branches. |
| Outcome: | Experiments on real-world projects show that GLMTest improves branch accuracy from 27.4% to 50.2% on TestGenEval benchmark compared with state-of-the-art LLMs, i.e., Claude-Sonnet-4.5 and GPT-4o-mini. |
DuET: Dual Execution for Test Output Prediction with Generated Code and Pseudocode (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that test output prediction is difficult to achieve due to code errors. |
| Approach: | They propose a framework that grounds prediction on error-resilient pseudocode and simulates execution via LLM reasoning to overcome limitations of direct execution suffering from code errors. |
| Outcome: | The proposed framework improves Pass@1 on LiveCodeBench, BigCodeBech-Hard, DevEval and HumanEval(+) and improves on pass@1 by 13.6 pp. |